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Abstract 

Comparisons of grade based learning outcomes between online and face-to-face course formats have 
become essential because the number of online courses, online programs and institutional student 
enrollments have seen rapid growth in recent years. Overall, online education is largely viewed by 
education professionals as being equivalent to instruction conducted face-to-face. However, the research 
investigating student performance in online versus face-to-face courses has been mixed and is often 
hampered by small samples or a lack of demographic and academic controls. This study utilizes a dataset 
that includes over 5,000 courses taught by over 100 faculty members over a period of ten academic terms 
at a large, public, four-year university. The unique scale of the dataset facilitates macro level 
understanding of course formats at an institutional level. Multiple regression was used to account for 
student demographic and academic corollaries—factors known to bias course format selection and grade 
based outcomes—to generate a robust test for differences in grade based learning outcomes that could be 
attributed to course format. The final model identified a statistical difference between course formats that 
translated into a negligible difference of less than 0.07 GPA points on a 4 point scale. The primary 
influence on individual course grades was student GPA. Interestingly, a model based interaction between 
course type and student GPA indicated a cumulative effect whereby students with higher GPAs will 
perform even better in online courses (or alternatively, struggling students perform worse when taking 
courses in an online format compared to a face-to-face format). These results indicate that, given the large 
scale university level, multi course, and student framework of the current study, there is little to no 
difference in grade based student performance between instructional modes for courses where both modes 
are applicable. 


Introduction 

By almost every measure, online education continues to play an increasing role in higher 
education. A recent survey by Allen and Seaman (2013) of over 2,800 universities and colleges found that 
32% of students—over 6.7 million total—are taking at least one online course. While the annual growth 
rate of online enrollment has slowed from an extremely rapid 30%+ rate of ten years ago, it is still 



increasing at a rate of over 9% every year. This growth rate is not surprising since almost 70% of higher 
education academic leaders believe that online education is crucial for their long-term strategy. Although 
historically many institutions met online education with resistance, over the past 10 years, the percentage 
of academic leaders that rate the quality of online classes equivalent or higher than face-to-face classes 
has steadily increased from about 52% to 77%. 

Meta-analytic work spanning the past several decades has identified negligible to modest 
differences in student performance between online and face-to-face course formats (Bernard et al., 2004; 
Zhao et al., 2005; Tallent-Runnels et al., 2006; Means et al., 2009). These meta-analyses do indicate 
significant variation between studies related to study design approach, study objectives, and measured 
student based learning or performance outcomes. To the extent that online courses are different than face- 
to-face courses, the grade students receive (i.e. performance) compared to the identical course taught in 
the alternative format should differ. There is a distinct need in the literature for larger scale studies to test 
these hypotheses that utilize single uniform institutional datasets composed of a variety of colleges, 
courses, and students. This paper compares grade based student learning outcomes in online and face-to- 
face delivery modes for a wide variety of subjects, courses, and course levels over a four-year period of 
time. 


Literature Review 

Within the academic literature there exists a considerable amount of disagreement about how 
effective online courses (compared face-to-face courses) are in achieving learning outcomes or 
educational objectives. Common study objectives within the literature include addressing a range of 
student based areas such as grade levels (A, B, C, etc.) and disparity, as well as student retention (course 
and institutional) and matriculation. Most of these studies look at comparisons of online and face-to-face 
courses taught by one faculty member in one subject at one particular institution. These studies are 
extremely important as they indicate local scale levels of variation among students; that said, smaller 
scale studies are not able to suggest institutional level conclusions. Many of these studies are indexed and 
described in the “No Significant Difference” literature concatenated by Russell 
(http://nosignificantdifference.org/). 

The majority of studies find that there is no difference in grade based student learning outcomes 
between modes of instruction. For example, Ashby et al. (2011), found no statistical differences in student 
grades in a developmental math course when taught face-to-face compared to online or blended teaching 
methods. Similarly, Larson (2009) indicated no statistical difference in grades of students taking an 
introductory management course. Earlier work by McLaren (2004), also indicated grades of online 
students in a business statistics course were not significantly different between students completing online 
compared to face-to-face course offerings. More recently, Driscoll et al. (2012), analyzed grade 
differences between students taking three online and face-to-face sociology courses. While Driscoll et al. 
(2012) initially found grades of online students to be significantly worse than their face-to-face peers, 
after controlling for aptitude (GPA), grade differences between the two course formats were statistically 
insignificant. Specifically pertaining to larger scale studies, Atchley et al. (2013) utilized a sample of over 
5,000 students from a variety of disciplines at a small public school and concurred with these other 
smaller scale studies that there were no statistically significant differences between student grades of 
online compared to face-to-face courses. 

Some studies have found that online methods of instruction can have negative influences on 
learning objectives and student grades. For example, numerous studies specifically identify lower grade 
performance in online compared with face-to-face course sections (Jaggars et al., 2013; Xu & Jaggars, 
2013). Increased grade disparity among students has also been associated with online courses, whereby a 
larger percentage of student grades of A’s, D’s and F’s are present in online courses where more B’s and 
C’s are associated with traditional face-to-face course formats, which suggests increased variation related 
to bimodal distributions (Atchley et al., 2013). Jaggars et al. (2013) point out that online courses seem to 
exacerbate achievement disparities that can occur between students taking face-to-face courses and 



parallel reduced course completion rates. Similarly, Xu and Jaggars (2013), in a large sample of students 
enrolled in 34 two-year public Washington State community colleges, found that students ultimately 
perform worse in online courses and were also more susceptible to dropping out compared to students in 
face-to-face formats (Xu & Jaggars, 2013). Relative to performance outcomes on short vs. long term 
scales in online and face-to-face courses, Johnson and Mejia (2014) indicate that while higher grade 
variation and lower completion rates are present in online modes, those students that do complete an 
online course tend to have higher program completion rates. Shea and Bidjerano (2014) also found that 
after controlling for student attributes, those students taking courses in online formats actually maintain 
higher matriculation rates to graduation. 

Course format modes are often found to significantly relate to particular student populations and 
demographic attributes (Stewart et al., 2010). These corollaries indicate a need to understand student 
attributes to appropriately describe trends between online and face-to-face courses. In addition to the 
grade similarities between online and face-to-face format modes, Ashby et al. (2011) described particular 
demographic corollaries of course formats. For example, older students and female students in general 
were more likely to enroll in the online course sections, but minorities were more likely to enroll in face- 
to-face course sections. Larson (2009) also found that a larger percentage of minorities and male students 
enroll in face-to-face course sections. Driscoll et al. (2012) found that students taking online courses tend 
to be older and added that online students also tend to have lower GPAs and work a greater number of 
hours per week outside of class. Relative to student preparedness as a description of students, Xu and 
Jaggars (2013) suggest that preparation levels between students taking online courses and students taking 
traditional face-to-face courses may be different, particularly between subject areas. 

Some studies have attempted to explain why students in online classes may have different grade 
based learning outcomes from students in face-to-face classes. Calafiore and Damianov (2011) suggest 
that one reason students taking online courses perform worse than their face-to-face peers is due to 
differences in course participation habits. Using a data set from five online economics and five online 
finance courses, Calafiore and Damianov (2011) found that the higher the student’s GPA and the more 
time students spent doing online course work, the better their performance in online formats. Differences 
in student conduct have also been suggested as a potential explanatory mechanism. However, Hollister 
and Berenson (2003) provide evidence that the online course format is not necessarily associated with an 
increase in academic conduct issues (e.g. cheating) compared to face-to-face formats. Moreover, Hollister 
and Berenson (2003) found no significant difference in students’ grades when taking proctored in-class 
exams versus online non-proctored exams. Similarly, Hallock et al. (2003) examined the performance of 
75 undergraduate business students and found no significant differences in learning styles and ultimately, 
the grade received by online students. They also found no statistically significant relationships between 
age, gender, or race with learning styles of online students. 

The objective of the present study was to incorporate a large data set of online and face-to-face 
course outcomes from a single university spanning multiple terms while controlling for often confounding 
demographic and academic corollaries to generate a robust test for differences in grade based learning 
outcomes as pertaining to course format. 

Methods 

To investigate online vs. face-to-face grade differences (student performance), course level data 
were obtained from a large Midwest public university. Course level data included mean student values for 
academic (online or face-to-face course format, course GPA, student credit hours, student GPA, college) 
and demographic (gender, minority status, age) variables. Since the focus of this study is on the 
comparison of grades between online and face-to-face courses, only courses that were taught in both 
formats (at least once) by the same instructor were included. The sample therefore consisted of every 
course taught at the institution over a three year time period that was taught in both an online and a face- 
to-face format at least once by the same faculty member over this time period. For example, if Professor 
‘Smith’ taught ENG 101 once (or more) in both an online and face-to-face format during this time period. 



then all sections of ENG 101 taught by him over this time period were included in the sample. If 
Professor ‘Smith’ did not teach any ENG 101 sections in either an online or face-to-face format, then 
none of his or her courses would be included in the sample. The suite of academic and demographic 
variables was included to serve as a control for parsing out variation actually attributable to a difference 
in course delivery mode. These academic and demographic variables have been supported in the literature 
to strongly covary with academic performance. Thus, these variables should be factored into any analysis 
involving student performance as a series of covariates to avoid selectivity bias. For example, 
nonminority, older, female students and students with higher GPAs are more likely to enroll in online 
courses (Cavanaugh, 2005; Xu & Jaggars, 2013). Also, in general, it is more likely that students with 
higher GPAs will achieve higher scores in classes regardless of the mode of delivery. If these variables 
were not included in the model, then a finding that online courses are associated with higher grades could 
be confounded by a particular demographic or academic attribute of the students enrolling, and not 
because the course was being taught in an online vs. face-to-face format. Other variables may also exhibit 
direct or indirect relationships with the variable of interest. Future analyses should incorporate these given 
data availability. 


Table 1. Mean student course attributes combined and separated by instruction type. 


Variable 


Means 



All 

Online 

F2F 

Observations 

6012 

1997 

4015 

Student GPA 

3.15 

3.41 

3.02 

Student Credit Hours 

74.25 

72.54 

75.11 

Gender (Male) 

0.43 

0.39 

0.44 

Minority 

0.22 

0.20 

0.23 

Age 

26.98 

30.00 

25.48 

CEHS Courses 

— 

621 

660 

COB Courses 

— 

240 

356 

COLA Courses 

— 

473 

1764 

CONH Courses 

— 

223 

248 

COSM Courses 

... 

207 

580 

CECS Courses 

... 

233 

404 

OTHER Courses 

... 

0 

3 


Ordinary least squares regression (OLS) was used to test for the effect of course type on overall 
course GPA. Ordinary least squares regression is a technique useful for assessing the relationship between 
an x and y variable where a linear relationship is expected (e.g. grades). Ordinary least squares regression 
has been used in previous analyses assessing student performance in online vs. face-to-face course types 
(Driscoll et al., 2012). Of particular use in this context is the ability to run multiple models gradually 
including explanatory variables to predict student performance. The ordinary least squares approach 
facilitates the recognition of direct and indirect corollaries of student performance in a given course type. 
Our initial model utilized a single term approach explaining course GPA as a function of online vs. face- 
to-face course type. Five models of increasing saturation were then used to parse out the amount of 
variation explained by other academic and demographic variables irrespective of online vs. face-to-face 
course types (Driscoll et al., 2012). Given the many direct and indirect relationships associated with these 
types of data, the issue of multi-collinarity should be addressed. Multi-collinarity is a commonly 
occurring problem when conducting regression analysis that can lead to erroneous conclusions. Multi- 



collinearity associated with independent variables was assessed using variance inflation factor values 
(Fox & Weisberg, 2011). To reduce non-essential multicollinarity between first order variables and their 
respective interactions (only one interaction included) a mean centered approach was used for student 
GPA (Dalai & Zickar, 2012). All analyses were implemented using the car package in the R statistical 
environment (R Core Team, 2013). 

Results 

The study included information from 140,444 students enrolled in 6,012 courses between 2010 
and 2013. The courses included 1,997 online and 4,015 face-to-face offerings. Students in online classes 
tended to be older in age, female, non-minority, and have higher academic GPAs despite similar total 
credit hours enrolled (Table 1). Overall, all independent variables except for minority status were found to 
relate to course GPA (Table 2). Our initial model to predict student grades as a function of course format 
was positive and highly significant, explaining approximately 39% of the variability in course 
performance (Table 2). This suggests that students in online courses will receive a grade point average 
that is 0.39 points (almost 40% of a letter grade) higher than a student taking a face-to-face course (Table 
2 ). 

Table 2. Regression models describing sources of variation in student performance. 


Independent and Control 
Variables 

Model 1 

Model 2 

Model 3 

Model 4 

Model 5 

Course type (online) 

0.390*** 

0.157*** 

0.097*** 

0.100*** 

0.066*** 


(0.020) 

(0.010) 

(0.018) 

(0.018) 

(0.018) 

Student GPA 


0.554*** 

0.464*** 

0.410*** 

0.373*** 



(0.018) 

(0.014) 

(0.014) 

(0.015) 

Student credit hours 


0.0004** 

0.0003* 

0.001** 

0.001*** 



(0.0001) 

(0.0002) 

(0.0002) 

(0.0002) 

Gender (male) 



-0.329*** 

-0.249*** 

-0.251*** 




(0.028) 

(0.029) 

(0.029) 

Minority 



-0.019 

0.053 

0.042 (0.038) 




(0.037) 

(0.038) 


Age 



0.018*** 

0.013*** 

0.013*** 




(0.001) 

(0.001) 

(0.001) 

CEHS (CECS) 




0.288*** 

0.277*** 





(0.032) 

(0.032) 

COB (CECS) 




-0.160*** 

-0.169*** 





(0.032) 

(0.035) 

COLA (CECS) 




-0.041 

-0.044 (0.028) 





(0.030) 


CONH (CECS) 




0.107** 

0.060* (0.04) 





(0.040) 


COSM (CECS) 




-0.090** 

-0.100** 





(0.033) 

(0.040) 

OTHER (CECS) 




0.3371 

0.342 (0.344) 





(0.346) 


Course type x Student 





0.166*** 

GPA 





(0.030) 


Intercept 3.02*** 1.36*** 1.303*** 1.525*** 2.784*** 


(0.01) (0.04) (0.048) (0.060) (0.051) 

0.06 0.29 0.32 0.35 0.36 


Adjusted R2 




This result is contrary to that of Xu and Jaggars (2011; 2013) who find students’ grades are lower 
when taking online courses. However, inclusion of additional variables in subsequent saturated models 
indicated the majority of this variation was a product of other academic and demographic parameters 
rather than course delivery mode (Table 2). The final saturated model indicated that the majority of 
course grade variation can be attributed to a student’s GPA. Our results indicate that students with higher 
GPAs tend to earn higher grades in courses. The final saturated model (Model 5) also showed an 
interaction between course type and student GPA indicating that students with higher GPAs tend to 
preferentially select face-to-face courses, and this combination results in the higher course grade. 
Ultimately, the overall effect of delivery mode in the final saturated model was found to explain less than 
0.07 points of variation in course GPA 

Discussion 

The overall result of little to no difference between course offering formats is consistent with the 
“No Significant Difference” literature (http://nosignificantdifference.org/). The significance and sign of 
the demographic variables were also generally consistent with other studies (Driscoll et al., 2012), which 
is namely that non-minority, older, female students have higher grades than minority, younger, male 
students. Of particular interest are the results elicited by interpreting the interaction term within the final 
saturated model. The interaction between course type and student GPA indicates that students with higher 
GPAs will perform even better in online courses (or alternatively, struggling students perform worse 
when taking courses in an online format compared to a face-to-face format). This has implications for 
student success and advising principles when identifying specific cohorts for online and face-to-face 
education. 

Although this study finds grades, independent of demographic and academic corollaries, to be 
practically similar regardless of the method of instruction, the sample used was from just one institution 
and only included courses that could be taught in both formats (e.g. excluded science courses with lab 
components). Future studies should incorporate additional academic and demographic variables wherever 
possible to expand the scope of the study. Also, additional universities from the United States as well as 
elsewhere in the global academic community should be included in future models to account for regional 
differences in instruction. Furthermore, future studies should separate course types by subject to identify 
intra-institutional variation in grade performance. Course level grade variation should also be more 
thoroughly quantified to ensure interactions between student grades do not become exacerbated with 
online vs. face-to-face course format selection. Course completion and program matriculation become 
essential pieces of assessment and represent two aspects of scale to provide a clearer short and long term 
measure of the impact of online vs. face-to-face instruction. 

Conclusion 

The landscape of higher education has been the subject of much discussion with recent 
technological advances. Although teaching technologies and methodologies are always changing, the 
rapid and ongoing growth of online teaching suggests that this format will be increasingly used in the 
future (Aslanian & Clinefelter, 2013). Of course, widespread adoption of any technology or methodology 
does not itself serve as a panacea for higher education. Changes to instructional formats should always be 
supported by rigorous studies that identify both improvements and shortcomings of each new or old 
instructional mode relative to learning outcomes. Ultimately, as online education programs continue to 
grow, success and failure of one format or another should be gauged through instructional content, grade 
based outcomes, grade disparity, course completion rates, program matriculation, as well as qualitative 
aspects of higher education. 
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